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Phenomenon of Interest 



The basic design problem 

Design a network to extract desired information about a 
given discrete/continuous phenomenon witii higii 
fidelity, low energy and low system cost 

Questions: 

- Where should the data be collected 

- What data and how much data should be collected 

- How should the information be communicated 

- How should the information be processed 

- How can network configuration be adapted in situ 

Single tool does not answer all questions 



Multiple Tools for Specific Instances 



Problem broken down into smaller parts 

- Coverage and deployment: Geometric optimization 

- Data fusion: Information theory, estimation theory, 
Bayesian methods 

- Communication: network information theory, geometric 
methods, Bernoulli random graph properties 

- Energy performance: Network flow analysis, linear 
programming 

- Configuration adaptation with mobile nodes: adaptive 
sampling theory, information gain 

• Not a comprehensive list 



Communication 
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Network Capacity and Node Density 



[Gupta and Kumar 2000] 



A simple model for a wireless 
network: 

- Disk of area A 

- N nodes 

- Each node can transmit at W bps 

Transmission range: disk of radius r 

- r depends on transmit power 

- Interference range (1+d)r 
Communication is successful if 

- Receiver within r of transmitter 

- No other interfering transmission in 
(1+d)r' of receiver 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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How many simultaneous transmissions? 



N nodes in area A 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 




Shaded circles represent interference range 

ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 



12 



Maximum simultaneous transmissions 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 




R,R2> dr, 

R,R2 > dr2 • 

Add the two inequalities 

ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 



At receiver R^, radius 
dr/2 is disjoint 
from any other 
reception 



T^ 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 



N/2 

'^(l/4){n(dr-f}<A 

i=l 




ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Maximum simultaneous transmissions 



Rewrite: 



N/2 

'^(l/4){ii(dv-y}<A 

i=l 

N/2 

2^ri2<16A/7rd2 

i=l 



Convexity: 



1 rN/2i2 . N/2 

■- i=l -■ 1=1 



Convexity: 




f(xi)/2+f(x2)/2 > f(xi/2+X2/2) 



r^ is a convex function 



ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 



17 

Transport Capacity 

• Transport capacity per transmission is W*rj (bit 
meters/second) 

• Total network transport capacity: 

N/2 

^Wri<W[8A/7iNd2] 



1/2 

i=l 



Per node transport capacity: 0(V1/N ) 

It can be proved that this is achievable as well 
- Requires multi-hop operation 



ACK: Slide adapted from PR Kumar's plenary talk at Sensys 2004. 
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Design Implications 

• Large network with arbitrary communication pattern is NOT 
scalable (per node capacity is diminishing) 

- Multi-hop operation can achieve the 0(VT7n ) 

• How to build a large network? 

1 . Use multi-user decoding: interference is also information! 

• Active area of research 




2. Transmit only a relevant function of the data: in-network 
processing 

3. Exploit correlation in data at multiple transmitters 

4. Large systems use hierarchy: add infrastructure 



19 



In Network Processing 

[Giridhar and Kumar 2005] 

Random multi-hop network, one fusion node 

- certain class of functions (mean, mode, std. deviation, 
max, k-tii largest etc.) can be extracted at rate 0{1/log(n)} 

- Exponentially more capacity than without in network 
processing: 0(1 /n) 



Network protocol design 

- Tessellate 

- Fuse locally 

- Compute along rooted tree of 
cells 



Fig. Ack: PR Kumar 
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Routing: Exploit Data Correlation 

[Luo and Pottie 2005] 

Each sensor generates measurement of data size R 

Due to correlation, extra information at additional sensor 
has data size r 



R ^^tL^^3LR^3r 






Ca = ^RD 

Ct = RD + ArD + ARd + 6rd 



Fig. Ack: Huiyu Luo 



Cc = RD-\- ArD + 4Rd 
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Data Fusion 
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Fusing multiple sensors helps 



Camera 1 




/ ■ 

Estimated target location 



View of target in image 
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Fusing multiple sensors helps 



Camera 1 




Camera 2 
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Fusing multiple sensors helps 



Camera 1 




Camera 3 



How much do 
additional 
sensors help? 



Camera 2 
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Resort to Information Theory 



Model the information content of each sensor 

Measure the combined information content of multiple 
sensors assuming best possible fusion methods used 

- Eg. Compute the distortion achieved for a given data size 

Can then determine if the improvement in distortion is 
worth the extra sensors 
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Measuring Information 



• Information depends on randomness 

• Two balls: P^ed = 0.5, P^iue = 0-5 



Let message be: 

Red ball is chosen = 0, 

Blue ball is chosen = 1 

Message has 1 bit of information 




Let 

Red ball is chosen = RED, 
Blue ball is chosen = BLU 
Still only 1 bit of information 
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Measuring Information 



• Four balls: 



green 



yellow 



Let 

Red = 00, Blue = 01, Green = 10, Yellow = 1 1 



Which ball is chosen = 2 bits of information 
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Measuring Information 



Four balls, but unequal probabilities: 



* "^red ~ ^-^J "^blue ~ O.ziO, Ppj^ppn ~ 0.1 ZO, r^,^ii^,,, - 0.1 ZO 



green 



yellow 



Save bits on more likely cases: 

Red = 0, Blue = 10, Green =111, Yellow = 110 

Average number of bits to communicate: 
0.5*1 + 0.25*2 + 0.125*3 + 0.125*3 = 1.75 
(over many-many balls) 

Which ball is chosen = 1.75 bits of information 
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Measuring Information 



• Number of bits measured by ENTROPY 

• H = -Zp*log2(Pi) 



Average number of bits to communicate: 
0.5*1 + 0.25*2 + 0.125*3 + 0.125*3 = 1.75 
(over many-many balls) 

Entropy: 

-[0.5*log(0.5) + 0.25*log(0.25) + 0.125*log(0.125) 

+ 0.125*log(0.125)] = 1.75 
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Example: English alphabet 



Ignoring case, we have 27 
characters 

- Naively: need 5 bits to 
represents a character 

• 24 = 16 < 27 < 2^ = 32 

Alternatively: measure the 
probabilities 

- calculate H « 1 .5 bits 

Efficient methods to 
automatically learn the 
probabilities and 
compress a file are 
available 

- Eg: Zip programs 
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Fig Ack: Mackay 2003 
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Conditional Entropy 

What if only incomplete information is available? 



Box 1 



Box 2 



Let P,,H = 0.25, P,„„ = 0.25, P 



blue 



green 



0.25, Py,ii„^ = 0.25 



Given Box 1 is chosen: 

P(red|Bl) = 0.5, P(blue|Bl) = 0.5, 
P(green|Bl) = 0, P(yellow|Bl) = 

Now, which ball is chosen = 1 bit of information 
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Conditional Entropy 

Suppose variable y has partial information about x 



Boxl 



Box 2 



H(x|y=a) = -Z p(Xi|y=a)*log2(p(Xi|y=a)) 

Given Box 1 is chosen: 
P(red|Bl) = 0.5, P(blue|Bl) = 0.5, 
P(green|Bl) = 0, P(yellow|Bl) = 

H(color|box=bl) = 0.5*log(0.5) + O.5*log(0.5) 
+O*log(0) + O*log(0) = 1 
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Conditional Entropy 

Entropy can also be defined for conditional probabilities 
- Take weighted average with probabilities of y 



Boxl 



Box 2 



H(x|y) = -I, p(y=a) H(x|y=a) 

Given Box 1 is chosen: 
P(box=Bl) = 0.75, P(box=B2) = 0.25, 

H(color|box) = 0.75*H(color|box=Bl) + 0.25* 

0.5*H(color|box=B2) 

= 0.75*1+0.25*1 = 1 



Conditional entropy = 1 bit of information 
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Example: Unreliable hard disks 

Conditional entropy helps model noise or channel error 
Given a hard disk with error probability = e 





(1-e) 

• Given e = 0.1 

• Suppose: want more reliable HDD 



Ack: Example adapted from Mackay 2003 



35 



Example: Unreliable hard disks 



Two Options: 

- Technology approach: Invent better storage, use more 
expensive components 

- Systems approach: build a reliable system around the 
unreliable HDD 



Simple Strategy: Store 
every bit 3 times - 000, 111 

- if one bit corrupted, can still 
fix it 

- Error probability: 
3V(1-e) + 0^=0.028 

t 



Redundant Array 




2 bits in error, 

3 possible ways 



all 3 bits in error 
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Example: Unreliable hard disks 

Not happy with 2.8% error 

- Want error probability: 10'''^ 

- Using simple strategy, need 60 disks 

• Not good for embedded design! 



Really Redundant Array 









Alternative: use conditional entropy analysis 
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Example: Unreliable hard disks 



Recovered data has errors 

- Has partial information about original data 
Denote 

- Y = recovered data 

- X = original data 

HDD communicates: 

- Information in X - information still left in X given Y 

- H(X) - H(X|Y) 

• Suppose source was Pred"Pbiue"0-5, so H(X)= 1 bit 

• From error model know: P(X|Y). Gives H(X|Y) = 0.47 

- HDD returns 0.53 bits for each bit stored 

• Two HDD's suffice! (can store 1.06 bits > 1 bit) 
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Example: Unreliable hard disks 

Recovered data has errors 

- Has partial information about original data 
Denote 

- Y = recovered data 

- X = original data 

HDD communicates: 

- Information in X - information still left in X given Y 

- H(X) - H(X|Y) 




CAVEAT: This holds only when a very large amount of data is 

stored. 

Simple repetition strategy does not work- better coding needed. 
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Mutual Information 



The quantity H(X) - H(X|Y) is l<nown as IVIUTUAL 
INFORIVIATION 

Written as l(X;Y) = H(X) - H(X|Y) 

- Helpful for measuring "recovered information" in many 
problems 

- Eg. Capacity of a channel (such as a hard disk or a 
wireless channel) 

• Max l(X;Y) 

P(X) 
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Rate Distortion 



• Suppose the HDD had zero error probability but we 
stored more data than its capacity 

- Can think of it as storing with loss due to error: again 
characterized using mutual information 

- Using lesser bits than information content in original data 

• Eg. JPEG, MPS 

• Given a distortion constraint D, how much data, R, is 
required to be stored: 

- R(D) = min l(X;X') 

• X' is such that it satisfies the distortion constraint and 
minimizes the mutual information 
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Gaussian Phenomenon 



Rate distortion function has been calculated when X is a 
Gaussian random variable 

- X = Gaussian with mean = 0, std dev = a 

- R(D) = (1/2riog(a2/D) 

• for D < a^. If tolerable distortion more than phenomenon 
variance, no need to send any data 



j 


X 

L 


R 









X' 



Proof: [Thomas and Cover] 
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Effect of Sensing Noise 



Gaussian Noise 




A 



X = Estimate ofX 



Phenomenon 



For given data rate, what's the distortion achievable? 

- R(D) measure with noise: 

- R(D) = (1/2riog[a4/(a2D+ a/D-a^a/)] 

• When positive, else zero rate 
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Quantifying the Fusion Advantage 




Phenomenon 



N. 




-*i + 



K 




M + 



N. 




-*i + 



Vi 



Y, 



Encoder 



Encoder 



Encoder 




A 

X 



Sensor 
Readings 

Characterize the distortion achieved with given rate when multiple 
sensors used 

- Distortion advantage gives the benefit of additional sensors 
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Quantifying the Fusion Advantage 

[Chen et al. 2004] 

• The total rate generated at multiple sensors is related to 
the distortion in estimation as: 




- where L is the number of sensors, o ^ is the variance of 
X and a ^ is the noise variance 
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Design Implications 

When all sensors have same noise, fixed D 



o 



0.664 




This is for 
Gaussian 
noise and 
Gaussian 
phenomenon 



Number of Sensors 



Using multiple sensors helps 
- Returns diminish beyond a small number of sensors 
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Another application of IVIutual Information 

[Grocholsky 2002] 

Mobile sensor: Having 
taken a measurement, 
where to take the next 
measurement? 

Among all possible motion 
steps, choose one which 
maximizes mutual 
information between 
phenomenon and 
measurement 

- Assume that density of 
phenomenon and noise 
model for sensor known 

Also extended to teams of 
mobile sensors 




x 
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Coverage and Deployment 
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Worst-case Coverage 



[Meguerdichian et al, 2001] 



Minimal exposure path 

- Likelihood of detection depends on distance from sensor 
and time in range 
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Sensing Model 



Sensing model 5 at an arbitrary point p for a sensor s 




where d(s,p) : distance between the sand p 

constants X and /fare technology and environment dependent 



Slide adapted from Meguerdichian, Mobicom 2001] 



50 



Intensity Model 



Effective sensing intensity at point p in field F : 



Aggregate of all 
sensors 



n 



IAF,p) = Y,S{s,, p) 



(Can generalize for fusion based metrics) 



Closest 
Sensor 



^min ^^m^S d(s^ , f) < d(s, f) \/S E S 



IciF,p)^S{s.,p) 



(Can generalize for k closest sensors) 



Slide adapted from Meguerdichian, Mobicom 2001] 
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Definition: Exposure 



The Exposure fov an object C^in the sensor field during 
the interval [t^,t^ along the path p(t) is: 



E{pm.h)t ']l{F,pit))^ 



dt 



jviethods available to compute this for a given network 
(using grid approximations and graph search techniques) 



Slide adapted from Meguerdichian, Mobicom 2001] 
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Average Coverage 



[Cortes et al, 2005] 



Quantify coverage with respect to phenomenon 
distribution in covered region 

- sensors close to phenomenon contribute more to coverage 




9 = sensor 

Shade represents 

phenomenon 

density 
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Average Coverage 

Region to cover: Q 

- q is a point in region Q 
Set of sensors: Pj, 1=1,..., n 

Coverage function: f, depends on distance from sensor 

- Coverage at point q due to sensor Pj is f(|q-Pil) 
Average coverage then becomes 

- Considering best sensor for each point q in Q 



n(P) 




Integrate 
over region 
covered 



max /(IIq-P. 

f 



Coverage at point q 
due to best sensor 



)(p(q)dq 



Weight by 

phenomenon 

density 
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Example 



Suppose sensing quality 
declines with square of 
distance 

- This gives f(x) = -x^ 

Best sensor for a point q 
is thus the closest sensor 
to it 

- Pj responsible for points 
that are closer to Pj than 
any other sensor 

- Call such a region 
around Pj as the 
VORONOI cell of p: 




Voronoi cells 
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Example 



Average coverage for this f can be written as: 

- Integral over each Voronoi cell 

- Sum over all cells 



n 






q - Pif 4>{(l)d(l 



- Where V|(P) is the Voronoi cell of P| 

- Negative sign indicates that coverage reduces if distance 
from sensors is large 
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Planning Deployment 



• Choose sensor locations to maximize coverage: 



Eg. 1 : Minimize worst case coverage by maximizing the 
exposure on minimum exposure path 



E{p{t\t,,t,)t ]l{F,p{t)) 



dp(t) 



dt 



dt 



- Eg. 2: Maximize H(P) 
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Adapting Deployment at Real Time 

• Suppose nodes are mobile 

• Need distributed algorithm that allows nodes to change 
their locations in order to maximize coverage 

- Algorithm should use only local information and not the 
global view of the network 

• One method to compute optima: gradient descent 
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Gradient Descent 



Consider a function f 
of two variables- x, y 

Find the x,y wliere 
f(x,y) is minimal 

- start at a random x,y 

- Compute gradient at 
that (x,y): gives 
direction of maximum 
change, (steepest 
slope) 

- Change (x,y) in that 
direction 

May reach local 
minima 




X 
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Distributed IVIethod 



• Take the gradient of the average coverage 

• Gradient turns out to be distributed over Voronoi cells! 

^(P) = 2 I {q-lH)mdq = 2Mv.(p)(CM;^(p) -pi) 

- where M is the mass of phenomenon density over the 
Voronoi cell of P| and CIV! is the center of mass. 
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Distributed IVIethod 



Distributed Motion algorithm: At each node Pj 

- Find Voronoi cell 

- Move towards center of mass 
Automatically optimizes total network coverage 

- No central coordination needed 




Initial 



Gradient descent 



Final 



Fig. Ack: F Bullo, April 15, 2005 talk at UCB 
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Energy Performance 
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Maximizing the lifetime 

[Bhardwaj and Chandrakasan 2002] 




Fig Ack. M Bhardwaj, INFOCOM 2002 
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Problem Formulation 



Given 

- Set of data sources and intended destinations 

- Fixed battery levels at each node 

Find the best routing strategy: maximum lifetime for 
supporting required data transfers 

- Nodes can change their power level and choose best 
multi-hop paths 
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Energy Models 



d 






r\ 


Transmit Energy Per Bit 


Etx = 0^11+ oc2d" 


n = Path loss index 



Receive Energy Per Bit E^ = a^j 



Relay Energy Per Bit E,^|,y = aii+ajdn+a^z = a^+ajd" 



Sensing Energy Per Bit 


j^~^ ^ 


^sense ~ ^3 



Slide adapted from: M Bhardwaj, INFOCOM 2002 
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Example: a three node network 




• Ignore wasteful 


(loopy) routes, route choices are: 

Rq: 1 ^ B 
R^: 1 ^2^ B 
R2: 1 ^ 3 ^ B 
R3: 1 ^ 2 ^ 3 ^ B 


Min-hop 

Rq: 0.25 
R^: 
R2: 
R3: 


Min-Energy 
Rq: 

R^: Other 
R2: 1.0 Choices... 
R3: 


Optimal 

Rq: 
R-,: 0.375 
R2: 0.375 
R3: 0.625 



Lifetime 0.25 



1.0 



1.38 



Slide adapted from: M Bhardwaj, INFOCOM 2002 
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How to find the optimal choice? 

• The search over multiple route choices can be stated as 
a linear program (LP) 

- Efficient tools exist for LP's 
Objective : 

\F\ 

max t = 2_] U 

ti = time for which route rj is used, |F| = number of choices 
Constraints : 

tj>0 : 1 < j < IF 

\F\ 
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Solving the LP for large networks 

• The number of route choices, |F|, is exponential in 
number of nodes, N 

- LP becomes too complex for large N 

• Map to a network flow problem which can be solved in 
polynomial time 

- Allows finding the optimal route choices for a given 
network in polynomial time 
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Equivalence to Flow Problem 



Route Choices: number of routes is exponential in N 

3/1 1 3/1 1 



(0) 

0.375(3/11) 

0.375(3/11) 

0.625(5/11) 

1.375(11/11) 



3/11 



5/11 



s 



3/11 



3/11 




5/11 



Network Flow View: number of possible edges is polynomial in N 

3/1 1+3/11 3/1 1 3/11+ 5/1 1 

1^2= 8/11 
1^3= 3/11 

2^3- 3/11 

2^b" 5/11 

5/1 1 3/1 1 

Slide adapted from: M Bhardwaj, INFOCOM 2002 
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Polynomial time LP 

Objective : 

max /. 

Constraints ; 

Non-negativity of flow: 

Conservation of flow: 



Yl /■'•= E ^'^^ ie[2,Aq 



5e[i,JV+i] <fe[i,^+il 



Total sensor flow: 



y^. fid- ^ Ai - 1 

dG[2,N+l] se[2,N+l] 

Energy constraints: 



^ Jif=ri ^?^_lL1l x>£^ri m_lii „ , . . / 



de [1,^+1] 5e[l,^+l] Fornodfi 1 only 

d^i &^i 
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Sensing Lifetime 



[Kansal et al, 2005] 



If data sources not specified, but given 

- Phenomenon to be sensed 

- Distortion quality required 

Choose the relevant sensors and determine the routes 
for maximum lifetime 











Phenomenon 
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Choose closest sensor 



Small data size (since measurement quality is high) 
Large routing energy cost (possibly long route) 





Phenomenon 
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Choose sensor with smallest route 



Small routing energy cost 

Large data size (measurement quality is bad) 






Phenomenon 
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Collaborative sensing 



Use multiple sensors and fuse 





Phenomenon 
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Lifetime-Distortion Relationship 



How to determine the best choice? 

- Combine lifetime maximization tools with data fusion 
analysis 

Find the possible choices of sensors for required 
distortion performance 

- Using data fusion analysis 

For each choice, find maximum achievable lifetime 

- using network flow LP 

Choose the sensors and routes which maximize lifetime 
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Sensor Selection 



All sensors within sensing range of the phenomenon are 
potential choices 

- How much data generated at each sensor: possible 
choices for given distortion are given by the rate allocation 
region of the Gaussian CEO result 



R. 







R. 



R. 



Phenomenon 
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1 ^ 1 - 2-2^'^ 1 



X 
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a\ 
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V non-empty A C (1, ..., A^} 
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Energy Constraints for Chosen Sensors 

• Same as before, but written for the variable source 
sensor rates 

- Maximize t as before 

/.j>0, Ri>0, u>Q,ViJe {!...., N} 
Flow Conservation: 
J2 f'^- Jl fsi=Ri. iG {!,..., AT} 

Energy Constraints: 



t 






<Ei. ie{L...,N} 
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Conclusions 



Network design depends on multiple methods 

- Optimal methods sometimes yield distributed protocols 

- Provide insight into the performance of available 
distributed methods 

Saw some examples of analytical tools that may help 
network design 

Open problem: Oonvergence of separate solutions 



